The present invention relates to graphic image generation. In particular, the present invention relates to the generation of models for use in training simulators and the like. More specifically, the present invention relates to the generation of graphic images in which certain objects, whose area is a small percentage of the total image and yet of critical importance to the utility of the interactive graphic task being performed, are generated prior to use in the simulator and are provided with enhanced features to accurately represent the object.
High quality graphic image generation is used in various fields such as flight simulators, medical training and surgery, computer games, and engineering workstations, to name a few. It is imperative that these systems provide realistic images for the benefit of the user. These images should have as a minimum sufficient quality to correspond to the visual scene experienced by the user in viewing the objects directly with either optically aided or non-aided vision. The overall objective is to facilitate the teaching or game playing environment for the benefit of the user. The system goal therefore is to provide an immersive environment which is perceived by the user to be very like the visual appearance of the task as it would be performed in the real world.
The versatility of computers and emerging graphics display technology has led to the development of computer based training in which a range of tasks are presented through the visual medium. As may be expected, the closer the graphics are to those encountered in real situations, the higher the confidence level of the student and instructor in the value of the training conducted.
The generation of a graphic image by a computer relies on the existence of a model representation of the object which is being imaged. This model precisely defines the geometric and visualization properties of the object for graphic presentation. Depending on the view which is desired, the visualization process transforms the inherently 3-Dimensional representation into a two dimensional object. Appearance of the representation mimics that of the actual object as though it were being viewed through a virtual xe2x80x9cwindowxe2x80x9d whose position corresponds to that of the display medium.
The display medium itself presents limitations on the display of objects. The standard method of rendering an object is to transform it into a series of discrete picture elements (xe2x80x9cpixelsxe2x80x9d) which constitute the image of the object. Since these pixels have a very well defined and static size on the display, they impose a limit on the accuracy and detail of the image. This represents a distortion of the objects"" model and detracts from the realism of the display. The standard industry answer to this problem has been a continuing effort to decrease the size of the pixels so as to improve realism.
A second problem encountered is that the complexity of the visual scene may lead to poor performance by the graphics system. This either reduces the allowable richness of the visual environmentxe2x80x94to maintain realistic performancexe2x80x94or causes the expense associated with graphics hardware to rise, to allow performance expectations to be met. The increasing speed of graphics chips is steadily driving the price of graphics systems down and enlarging the scope of applications which can be handled by training systems.
The limitations of the display constitute the problem to be addressed. The following kinds of problems are present in known displays:
1. Aliasing: The image appears to be composed of xe2x80x9cblocksxe2x80x9d of a given size. Lines appear to be staircases, circles have sawtooth boundaries, etc.
2. Shape Distortion: The finite size of the pixels causes small objects to be represented by single pixels in which the aspect of the object cannot be determined.
3. Accuracy: The intensity of the object is not correctly computed and the object flickers due to random subsampling of the small object as successive frames are computed.
4. Special Effects: A lack of cues that the user may specifically rely on to assist in identifying or tracking the motion of an object. For example, sun glint from a windshield canopy.
A basic assumption is that only certain, very specific objects need to be rendered with high accuracy and detail. If an attempt were made to render the entire scene with additional accuracy, the graphic processor would be swamped with the requirement and the potential advantages of an improved rendering would not be available.
To this end, it is desired to provide systems which do not create false impressions with unrealistic or inaccurate object representations. For example, flight simulators are employed to train fighter pilots on how to quickly detect objects such as enemy planes and missiles. The pilot does this by scanning the horizon in a predetermined pattern along with other visual and auditory warnings. As such, if the flight simulator renders an object with a fluttering appearance or an unrealistic large size, a false impression of the target is generated. As such, the training exercise is detrimental in that the actual appearance of an enemy plane or target is unrealistic. Hence the visual expectations of the pilot in air combat become unrealistic and life threatening. Similar limitations exist for medical training and surgery preparation/execution and other similar applications. Hence accepted practice is to overcome this very serious limitation by resorting to alternatexe2x80x94and generally more expensivexe2x80x94means of accomplishing the objective. As an example, after training in a simulator, the pilot must spend a large number of hours in the aircraft to become familiarized with the appearance of aircraft and missiles in the real world.
Attempts at improving graphic image processors used with simulators and other interactive graphics devices continue due to the desire to improve the quality of displays. These are limited by the pace of advancements in computer and display technology.
In general, it is known to apply improved rendering techniques to an entire image display to enhance the overall appearance of the images presented. However, this approach rapidly consumes processing power available and accordingly, limits other operational aspects of the image processor such as real-time presentation of the total visual environment. Moreover, current technology graphic processors using embedded graphic algorithms are unable to selectively improve the visual appearance of those items whose detail is particularly important and critical to the overall success of the training simulation. This is exemplified by the aforementioned planes and missiles that require high acuity presentation in order to assure that the pilot is being trained in an environment as similar as possible to the visual environment likely to be encountered in actual air combat.
The present processing equipment does not prioritize these objects and accordingly, processes the important items as it would any background information. This limits the usefulness of the training or display environment.
One alternative to the aforementioned approach is to employ high acuity projectors in conjunction with a graphic image processor. This technique generates a simulated background scene and superimposes the critical images onto the scene with a higher resolution. This requires additional processing equipment and is quite expensive. Moreover, the high resolution projectors of today are unable to represent the critical objects with the acuity and real world appearance necessary for effective training or practice.
An extension of this approach is to provide a hardware-based solution utilizing high resolution Area of Interest displays. In conjunction with this, a mechanism is provided for tracking pilot head position and those areas where the pilot is perceived to be looking are processed with high resolution. Unfortunately, this method employs unrealistic background scenes which appear artificial and do not present an accurate representation for a training simulator. Hence the lower detail background image appears to the pilot undergoing training to be quite different than the remainder of the visual scene. This provides the pilot with a visual cue not available in air combat and lessens the pilot""s ability to perform air combat maneuvers effectively.
The current preferred system for generating graphic images for simulators is to provide an image database that is accessed by a graphic computing engine. Depending upon input from the trainee, images are rendered to a memory buffer and then displayed at about 60 frames per second. One enhancement to this current technology is to render the critical objects, such as enemy airplanes, in a separate memory buffer which is then transferred to the main memory buffer for display. Although an improvement, a high resolution display of about 5,000xc3x974,000 pixel screen density is required to properly display the critical objects. This approach is still quite costly as it still requires the use of laser projectors which are not currently available and whose cost is likely to be very prohibitive upon the initial introduction of this improved technology.
One alternative to the foregoing systems is disclosed in U.S. patent application Ser. No. 09/257,452 entitled xe2x80x9cVariable Acuity Rendering for a Graphic Image Processing Systemxe2x80x9d filed on Feb. 25, 1999, which is incorporated herein by reference. This patent application provides for the recognition of predetermined critical objects in a graphic display. The graphic image scene is enhanced by storing a plurality of object descriptions which are segmented into critical objects and background objects in an image database. The critical objects and the background objects are transferred to a computing engine which renders the critical objects and the background objects according to expert user recommendations. The critical objects are rendered separately with acuity enhancements to generate enhanced critical objects that greatly improve the realistic impression of the objects in simulators. The background objects and the enhanced critical objects are then transferred into a memory buffer as a graphic image which is then displayed at a single resolution. Although the forgoing system is an improvement in the art, it does require additional computer processing capability in order to accommodate the critical objects with high acuity.
Based upon the foregoing, it is evident that there is a need for a graphic image processor system which displays critical objects from an intelligent module library while using available graphics processing power to display the entire scene at the same resolution. The availability of this capability markedly improves human performance in real world visual tasks for which extensive training is currently the norm.
In light of the foregoing, it is a first aspect of the present invention to provide a method for generating an intelligent model library for a graphic imaging system, wherein an image is generated by a method of enhancing recognition of predetermined objects in a graphic display rendered by a computing engine.
Another aspect of the present invention is to provide the system and method, as set forth above, which includes a method step of determining processing system limitations such as the type of display system utilized, the type of image generator utilized, the object to be modeled, the expertise of the trainee, and the various types of backgrounds in which the object model will be displayed. This analysis is used to determine what affect, if any, the system limitations may have on the appearance of the picture elements or pixels that make up the critical object.
Still a further aspect of the present invention is to provide the system and method, as set forth above, wherein a model is selected as a critical object, wherein the model is constructed as a series of wire-framed polygons with specific attributes assigned to each polygon. As a result, the final object appears as an accurate representation of the actual object at a relatively close range.
Yet another aspect of the present invention is to provide the system and method, as set forth above, which includes a method for generation of an intelligent model library for use with a simulator, wherein the model is enhanced for various attributes of size, shape, and contrast, to name a few, so as to improve the acuity of the model when represented in a display at different distances.
Still another aspect of the present invention is to provide the system and method, as set forth above, which includes a method for generating an intelligent model library for use with a simulator, wherein the enhanced model is tested in very different scenarios to obtain an optimum desired enhancement of the base model.
A further aspect of the present invention is to provide the system and method, as set forth above, which includes a method for generating an intelligent model library for use with a simulator, in which the enhanced models are stored in an image database.
Still yet another aspect of the present invention is to provide a method for generating an intelligent model library for use with a simulator, as above, wherein the stored models are exported to a simulator program for use in a simulation.
Still an additional aspect of the present invention is to provide a method for generating an intelligent model library for use with a simulator, as above, wherein the enhanced models are adjusted to facilitate the trainee""s level of expertise.
The foregoing and other objects of the present invention, which shall become apparent as the detailed description proceeds, are achieved by a method for enhancing the appearance of predetermined objects in a graphic display, rendered by a computing engine, comprising determining system limitations of the computing engine for displaying an object in the form of a group of pixels, wherein the group of pixels distort representation of the object at a distant range within the graphic display; constructing a base model of the object viewable at a normal range within the graphic display; constructing at least one enhanced model of the object to provide a first set of attributes to the group of pixels to compensate for the distortions of the object at the distant range within the graphic display; and storing the base model and at least one enhanced model in an image database.
Other objects of the present invention are attained by a computer-readable medium having stored thereon a data structure, comprising a first field containing data representing a base model viewable at a normal range within a graphic display; and at least a second field containing data representing the base model viewable at a second range other than the normal range within the graphic display, the second field having a set of attributes represented by a group of pixels to compensate for any distortion of the base model at the second range.
These and other objects of the present invention, as well as the advantages thereof over existing prior art forms, which will become apparent from the description to follow, are accomplished by the improvements hereinafter described and claimed.